In the previous lessons, you looked at building a binary
classifier that predicted cats versus dogs or
horses versus humans. You also saw how
overfitting can occur and explored some practices
for avoiding it. The problem with these of course, is that the training data
was very small, and there's only so
many common features that can be extracted, even if we do some tricks
like image augmentation. But in both these cases, you built the model from scratch. What if you could take
an existing model that's trained on far more data, and use the features
that that model learned? That's the concept of
transfer learning, and we'll explore
that in this lesson. So for example, if you visualize your model
like this with a series of convolutional layers before dense layer leads
your output layer, you feed your data
into the top layer, the network learns
the convolutions that identify the features in
your data and all that. But consider
somebody else's model, perhaps one that's far more
sophisticated than yours, trained on a lot more data. They have convolutional layers
and they're here intact with features that
have already been learned. So you can lock them instead of retraining
them on your data, and have those just
extract the features from your data using the convolutions that they've already learned. Then you can take a model
that has been trained on a very large datasets and use the convolutions that it learned when
classifying its data. If you recall how
convolutions are created and used to identify
particular features, and the journey of a feature
through the network, it makes sense to just use those, and then retrain the dense layers from that model with your data. Of course, well, it's
typical that you might lock all the convolutions.
You don't have to. You can choose to retrain
some of the lower ones too because they may be too specialized for
the images at hand. It takes some trial and error to discover
the right combination. So let's take a well-trained
state of the art model. There's one called Inception, which you can learn
more about at his site. This has been pre-trained
on a dataset from ImageNet, which has 1.4 million images
in a 1000 different classes.